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1. INTRODUCTION 

Over the last few years, serious games have been effective educational tools which attracted the 
attention of students, educators and professionals, used in many areas of knowledge including military 
defense, health care, manufacturing, education, economy, religion, business, emergency management, 
politics, city planning, scientific exploration, engineering, tourism and medicine [1]-[6], etc. For instance, 
this diversity it’s not surprising because serious games can be designed to solve complex problems 
collaboratively [7], make business process more efficient, achieve predictive modeling and real time 
visualization [7], and more retention of knowledge compared to traditional methods. An example of 
educational decision is selection of courses in the higher education [8]. 

According to serious games definitions [9],[10],[11], we can resume the field in two term Fun and 
engagement. However, the serious side like learner assessment, recording user or learner data, enhancing 
results for professionals and so on presents a challenge for researchers according to their learning benefits 
and importance. For serious games prove their effectiveness; they must be able to transmit knowledge 
properly and allow measuring and analyzing the learning outcomes [12]. 

Traditionally talking, teachers or people in charge after every competence transfer they evaluate 
students learning achievement with exams. Besides, the field is used wisely in e-learning[13]-[16], and there 
are some generalized methods like assessment of student competence, assessment of student achievement. 
Assessment of learning and training requires a systematic approach to determine a person’s achievements and 
areas of difficulty. Generally, assessment method is designed to evaluate many competencies but the final 
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result is merely a grade without any description about student’s achievement in every competency [17]. 
Standardized assessment methods often take less time and are easier to administer, and their results are 
readily interpretable [18]. However, this standardized assessment method are useless for non-computer or 
science person. Indeed, we provide to present an assessment framework which can be explicit and easy for 
professional and non-professional people. 

In order to test the efficacy of this assessment plan, we will present our game entitled. Recruitment 
protocol that has as a purpose to train the player for a job interview. Obviously, the most important axis in the 
game is to judge the efficiency of learners and their abilities to adapt in the corresponding situation. A 
debriefing of the learner experience will be sent to the game tutor or an expert. The game is doted with some 
data mining algorithms. One of the challenges is to introduce an assessment plan in the game development 
process and face the instructors and experts how to evaluate the learning outcomes in order to identify 
students lacks for the given goal or field. 

In this perspective of research we aim to present, first the literature review about assessment, e- 
assessment their advantages, methods, tools and applications domains. Then, we discuss how to collect and 
to reuse data during the experience. In addition, we provide to present our framework by giving a significant 
example and we present the result of our proposal. Finally, we close by significant perspectives. 


2. LITERATURE REVIEW 

In our research of the assessment as scientific field, we find a number of definitions defining it, such 
as: ‘An ongoing process aimed at understanding and improving student learning; It involves making our 
expectations explicit and public; setting appropriate criteria and high standards for learning quality; 
systematically gathering, analyzing, and interpreting evidence to determine how well performance matches 
those expectations and standards; and using the resulting information to document, explain, and improve 
performance’ [15]. 

Also we find another definition that present the field ‘as the systematic basis for making inferences 
about the learning and development of students; More specifically, assessment is the process of defining, 
selecting, designing, collecting, analyzing, interpreting, and using information to increase students’ learning 
and development’ [13]. According to these definitions, we can summarize assessment as a process aimed to 
understand, analyze, collect, interpret and use the information to improve learning and development of 
learners. 

Generally speaking, assessment in an educational system comes in different aspects; Diagnostic 
assessment hopes to improve the learner’s experiences. Is often used before teaching or at the beginning of 
the experience (Assess what the learner already know). Formative assessment Contributes to learning through 
providing feedback [19]. It occurs prior to and during the course and allows adjustments if necessary. It plays 
a regulatory role and learning support [20]. Summative assessment is like diagnostic assessment but it 
appears at theend of the learner experience. It’s used to measures the learner’s achievements at the end of the 
experiences. This measurement is often based on exams, papers projects and presentations, portfolios which 
help to submitting student progress and collecting information about the student. Integrative assessment or 
authentic assessment is a reflective and iterative participatory process that links knowledge (science) and 
action (policy) regarding complex global change issues [21]. Integrative tasks can be for formative or 
summative purposes.Especially for serious games, the formative assessment is the more useful and should be 
used in the game process to be part of the experience. Although several methods and techniques have been 
used to assess learning in serious games [22] and simulations in general, summative assessment is commonly 
accomplished with the use of pre- and post-testing, a common approach in educational research [23]. 
However, summative, formative, diagnostic and integrative assessment works together to improve learning. 
They should be the important components in the game design and at the game level. 

In summary, assessment involves a rich infrastructure capabilities that enables us to evidence of 
student learning in a much deeper and often more authentic than what has been possible before with 
traditional methods. A number of software products are available for online educational testing and 
assessment [24]. However, the constructor can combine between a theory and technology to acquire desired 
result. In Table 1 we provide to present some of existing method and tools with corresponding description. 

A literature review was performed to identify different approaches and theories in order to integrate 
assessment into serious games. In our research we find also a multitude of papers discussing the different 
types of assessment and papers discussing assessment frameworks for serious games. Some studies cover 
formative and summative assessment. For example authors in [28] uses serious game for formal assessment. 
However, the assessment model used in this study based on Gagne’s nine events of instruction. Others , such 
as [29], are focused on learning skills and motivation which are assessed using a probabilistic approach 
performed by the user. Their approach is based on adaptability in order to help learners to increase their 
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motivation skills. Indeed, providing an automated tool for monitoring and analyzing the actions performed by 
the learners, authors in [30] have as purpose to use petri nets extracted from the expert behavior in the game 
and compare the learner interactions to it using several diagnosis indicators. 


Table 1. Assessment Methods and tools 
Type Description 
Assessment management 
systems 


Tools for instructors to administer and analyze tests 


Tool that keep trace of the learners‘ activity in the shape of events in 
game and basing on these ones assesses the related performances respect 
to a predefined set of competences[25]. 


A Competence 
Performance Analyzer 


Tools for natural language 
answer assessment 
Leap motion controller A small device that can be connected to a computer using a USB[26]. 
Eye Trackers measure the saccades (fast movements) and fixations 
(dwell times) of human gaze[27] 


Tool that analyze and assess answers written in text area[14] 


Eye Tracking 


Providing a list of theoretical questions, assessment matrix provides faculty members (especially 
ones with little assessment experience) with a structure for developing their assessment plan using a series of 
questions-as they answer the questions, they essentially articulate the plan [31]. Accompanying learners and 
their evaluation is a subject already treated in the literature[32], [33].The evaluation needs of serious games 
are particularly relevant to evaluation before apprenticeship, during apprenticeship (real-time help) and after 
apprenticeship (diagnosis of learned skills), possibly involving a debriefing between the learner and the 
monitor. 

To summarize, all highlighted studies are generally focused on game play session, learner 
performance or achievement, summative and formative assessment and learning skills and motivation. Also, 
Assessment matrix is focused on a rhetorical side of assessment plan. However, it become necessary to 
combine several of these good techniques to encompass the strengths of all the assessment techniques 
reviewed. Especially for serious games assessment plan should combine between serious and fun in order to 
attempt the pedagogical objective, Assessment methods must be recognizable within the game design, as 
Michel and Chen [10] states “Serious games like every other tool of education must be able to show that the 
necessary learning has occurred.” 


3. WHO’S IN CHARGE TO DESIGN AN ASSESSMENT MODEL FOR SERIOUS GAMES? 

Despite the abundance of literature on SGs, few papers provide a detailed description of the specific 
SG assessment design. Still, “many educational games do not properly translate knowledge, facts, and 
lessons into the language of games. This results in games that are often neither engaging nor 
educational” [34]. However, in this part of paper we provide to describe the roles of each author responsible 
of designing assessment in serious games. Persons who are in charge to design a serious game are: 
Professional expert, Pedagoge expert and Game designer expert. 


3.1. Professional expert: 
It is thus possible to reveal four criteria for measuring professional competence: 
Learner level of expertise in a specific field, 
A measure of his degree of knowledge in a given field, 
The degree of operationally of the action he is carrying out, 
The degree of variety of the reasoning he leads. 
The Figure 1 describes the check list that should be done by the professional expert. As mentioned 
in the Figure 1, expert should: 
a. Describe each situation and define their required competencies 
b. Define the thematic with their level of competence 
c. Classify action 
d. Describe actions and different scenarios of each action 


aoe 
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Activity 


Work situation ‘= Competence required 


Initiation 
mastery 
level of competence 3 
Theme _ Sub-theme . = deepening 
Domaine ‘€ Confirmation 
Professional - 
Theoretical knowledge 
Environmental knowledge 
Category of questions _ Procedural knowledge 
Ranking of questions =- Operational Expertise 
Cognitive Expertise 
Type of question 
Description of the situation with problems 
encountered during the experiment 
Figure 1. Professional expert 
3.2. Pedagogue Expert 


Aun: Oe 


Is the person who’s in charge to implement and to validate the learning transfer via the game 
experience. The Figure 2 describes the check list that should be done by the pedagogue expert. As mentioned 
in the Figure 2, pedagogue should: 

Question: Action 

Response: feedback (positive; negative) 

As mentioned in the figure, pedagogue should: 


Define the category of problem ( or question in quiz) 


How to implement the problem in order to ensure better learning achievement 


Define different errors types 


When can be the guidance triggered. When each step can we implement this problem (pre in or post 


game) 


Define educative message to show to the learner in order to complete his learning 


Category of question 


Type of question 


Complete the information (optional) 


Inconsistency 

Irrelevant information 

No reading ability 

Absence of analytical capacity 
Absence of reasoning ability 


Incomplete work 
Calculation error 


Helping the learner to correct mistakes 


States 
Question — Formuller 
Guidance 
if correct _ = 
Type of error 
Response — = 
Else _ 
Pedagogue —- 
Guidance _ 
Initiation 
mastery 
Level of expertise — deepening 


Confirmation 
Pre game 
In game 
Post Game 


Step —- 


(Self Assessment) 


indication au cas de besoin (facultatif) 


Assistance — 


Figure 2. Pedagogue expert 
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3.3. Game Designer 

Game designer is the person who’s in charge of gamifying instruction cited by both pedagogue and 
professional expert. He uses several tools and platforms for creating scenarios, object, sounds, and graphics 
and so on. Especially, for serious games game designer should have basics in different subject and pedagogy. 


4. ASSESSMENT PLAN DESIGN FOR SERIOUS GAMES 

In the light of our experience and according to the analysis made by our research team concerning 
the strong and weak point of each method, the method below, presents the proper integration of assessment 
plan in serious games design, supporting different methods and tools during the different steps of design and 
development process. The Table 2 sumarizes the main phases of development with their corresponding 
description. 


Table 2. Development Phases 


Background Iterative design phase Iterative development phase 


Prototype 
Background 
© 
Objectives: Pedagogical model: Are the educational outcomes met? 
Define overall objectives of the Design instructional strategies How can you convince them that the 
game What a specific data do you need to collect? educational outcomes were met? 
Identify goals for the desired Required assessment methods and tools How can you improve your program and your 
results of content Defining a competencies model? assessment process? 
Users: Evaluation Space Assess performance 
Typology of users Social Content Enhance retention and transfer 
Needs analysis Evaluate the pertinence of data collected 
Learner’s requirement game design 


Accessibility and usability standards 


4.1. Overall Objectives: 

Context layer: The expert or the game designer should situate the game in a specific context by 
defining domain, learning objectives, game intention, target audience and a brief description of summarizing 
the main objectives of the game as presented in Table 3. 


Table 3. Overall Objectives 


What are the overall objectives of the program? 


Domain: Health, business, finance, military, management, recruitment, communication and marketing, business teaching, social security, 
Construction, Creative industries, life sciences, information and communications technology, Tourism, food and drink ... 

Intention: Educative message broadcasting ,Informative message broadcasting , Marketing & Communication message broadcasting 
„Submit a subjective message ,Training ,Exchanging goods ,Tell a story ,Operating a license 

Student requirement: what are the learner requirements? 

Target Audience: Professionals, non-professionals, general population 

Brief description: Text 


4.2. Learning Outcomes 
Learning layer: The expert or the game designer should have clear idea about learning outcomes. He 
or she must give a corresponding answer as presented below in Table 4. 
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Table 4. Educational Outcomes 


What are the program’s educational outcomes? 


an ability to: 

Engage in life-long learning; 

Know the basics; 

Able to memorize (factual learning) ; 

sensor-motor skills; 

Adapt and transfer knowledge in a new context, use information, methods, and theories in new situations; 
Analysis of knowledge-based problem solving, prediction, projection conclusions, choice, argumentation.; 
understanding the social environment and others; 

Self-evaluation; 

Others... 


4.3. Learner Portfolio: 

Students’ experiences are collected throughout a program which is assessed by game engine using a 
predefined rules and a common scoring guide/stage. Portfolios may contain several informatione.g. 
“Research papers, reports, tests, exams, case studies, video, personal essays, journals, self-evaluations, 
exercises, etc”. Portfolio is considered like a direct assessment method. Table 5 summarizes needed data to 
collect. 


Table 5. Student Portfolio 


student's portfolio 


Learner: Description 

Type: Junior, Senior intermediate, expert 

Tutor: Teacher, doctor, director... 

Context: Open Space, Classroom, online... 

Result: Score, time, number of correct answers, number of false answers, number of redundant faults 

Note: Failed, medium, low, high, good, excellent 

Decision: If we use serious games with e-learning platforms e.g.’"moodle edx ...”, we can use students portfolio to complete data 
information 


4.4. Assessment Methods: 

As presented by [9] learning objectives are brief statements that describe what the learner will be 
expected to learn by the end of the game. While educators or professionals use learning objectives in 
different ways to achieve a variety of instructional goals across multiple developmental stages, grade levels, 
age and so on. Each program will select the assessment methods as presented in Table 6 that will provide the 
most useful and relevant information. When selecting which assessment methods to use, consider the 
availability of resources, the usefulness of the results and the learning outcomes. 


Table 6. Assessment Methods Needed to Collect Data 


What assessment methods will you use to collect data? 
Method Description 


Alumni Survey Can provide information about program satisfaction, preparation, employment status, skills for success. 
At the end of the game 
Type: Indirect 

Capstone Project Has as purpose to integrate knowledge, concepts, and skills acquiredby learner or the learner during the sequences 
of the game. 
In the game 
Type: Direct 

Certification These standardized tests are developed by outside, professional organization to assess general knowledge in a 
discipline 
Type: Direct 

Employer Survey Information to collect can include on the job skills, field specific information, etc. 
Type: Indirect 

Institutional Data Review both serious game and student data that is collected. 
Type: Indirect 

Observations Provide information on student behaviors and attitudes. 

Pre and post tests Typically an serious game is administered at the beginning and at the end of a course or program in order to 
determine the progress of student learning 
Type: Direct 

Game Review Reviewing a game involves determining if the objective is meeting the goals and outcomes that have been 
established 
Type: Indirect 
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4.5. Data Collection 

Data collection is used to organize institutional processes, analyze student learning gaps and 
implement strategies, allocate resources, and continuously evaluates the efficacy of the learning objectives to 
support and improve student learning. It presents a quantitative data collection that helps to evaluate the 
learner experience. However, these data are collected from Patients; Students, learners, Teacher or the 
former; Evaluator agent; Guidance agent. Finally, the game designer should describe detailed needed data 
from each actor. All collected data depends to assessment method chosen by the game designer. Generally 
speaking they can be resumed into three layers: Pre-game, In-game, Post-Game. There are many reasons to 
collect data such as learners guidance during the game, test learner efficacy, measure the learner 
achievement; enhance the student level, improve the level of learner, recruitment; health analysis and so on. 
Table 7 presents the main specific data needed to collect. 


Table 7. Specific Data to Collect 


What specific data do we need to collect? 


Name; Age ; Sex;Time;Score; Number of gestures; Number of right answers; Number of wrong answers. ; Check if the learner need 
assistance or help; 

If the game require reflection 

Number of redundant fault 

right action, too early, too late, suboptimal, erroneous and missing 

Others... 

These data depends on the nature of the game and it’s can be defined by expert or game designer 


The evaluation of the learners requires the collection of information from different sources 
according to the role incarnated by the learner in the scenario. The information required for evaluation thus 
varies in type and nature (depending on the role played by the actor), and must be acquired from sources of 
knowledge or data distributed over different software components. 

Interactions can be evaluated by the time elapsed between the detection of the problem and its 
resolution as well as the impacted zone. Each data required for evaluation can use a different model of 
representation, as well as a calculation of evaluation specified to produce a learning score to validate or not 
the learning objective of a scenario. This knowledge can be produced individually or collectively 
(interactions between actors and / or simulated players). 


4.6. Data Evaluation: 

In this section, we present how can we interpret data with respect to the existing methods as 
presented in Table 8. Data analytics is a science consisting of examining raw data, in order to draw 
conclusions from this information. It is used to verify theories or to refute existing models. In this perspective 
of study, is focused on the learner achievement. Data analysis is then used to determine if the systems in 
place effectively operate efficiently and achieve the learning outcomes goals. 


Table 8. Interpret and Evaluate Data 


How will you interpret and evaluate the data? 


Method Description 
Statistics Is one branch of mathematical science studying the collection, analysis, interpretation, presentation, and 
organization of data[35] 
Business intelligence "intelligence" encompasses IT solutions providing decision support with professional, chain-end, reports and 
monitoring dashboards activities [36] 
Web analytics Meaning the study of the impact of a website on its users[37]. 
Operational research A mathematical discipline that contract with advanced analytical methods to help deciders to make the better 
decision 
Artificial intelligence Data mining is a new concept in education domain. This concept can be helpful as a work of bridge between 
and data mining this lacks of knowledge [38]. It’s the process of analyzing data from different perspectives and summarizing 


the results as valuable information. It has been defined as the nontrivial process of identifying valid, potentially 
useful, novel and ultimately understandable patterns in data [39]. 


Social network Meaning the study of the impact of social networks on its users. It means also the study of collection, analysis, 
analysis interpretation, presentation, and organization of user’s data in social networks[40] 
Information The study of (interactive) visual representations of abstract data to reinforce human cognition. 
visualization 
Learning analytics The application fields of the learning analytics concern modeling of user knowledge, user behavior, and user 


experience, user profiling; modeling of key concepts in a domain and modeling a domain’s knowledge 
components, and trend analysis 
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4.7. Debriefing and Feedback 

After each programme, tutor or former should convince parent of learners that the learning 
objectives are achieved as presented in Table 9. Outcomes define what you want a serious game to 
accomplish. Generally speaking, learning outcomes statements may be considered to be exit behaviors. 
Outcomes achievement can be presented in post-game for student and tutor like a quiz to have more 
information and to help the game designer to enhance our game engine. Information about student learning 
outcomes assessment results will be available at the appropriate levels of granularity for use by programs, in 
analysis and evaluation, and decision-making. 


Table 9. Learning Achievement 


How can you convince them the objectives were met? 


Goals :Attend the aims of a game 

Outcomes:the changes in the lives of individuals, families, organizations, or the community as a result of the game. Evaluate intrinsic 
motivation 

Indicators:the specific, measurable information collected to track whether an outcome has actually occurred 


5. REVERSE ENGINEERING 

When applying assessment plan in design process and if the designer respect the instruction as 
presented above the result will be as present in the Figure 3. Evaluation type in serious games: Real time 
evaluation: This evaluation relates to a diagnosis evolving throughout the game (game scenario). Real-time 
evaluation refers to the evaluation of an action or mission (Behavior space). To evaluate an action, the 
calculation of a score is based on the corresponding indicators. 

Final individual evaluation: Individual evaluation is a summative evaluation that evaluates and 
certifies the learner's learning in the game scenario. This evaluation compares the state of the learner's model 
(learner actions As well as his/her knowledge) with the model of the domain and establishes a diagnosis on 
the skills acquired and the skills that remain to be learned. 

Collective evaluation: Using the Social Space, an interaction graph is constructed, representing the 
different communications and interactions between the different actors (learners as well as simulated actors). 
With this graph, we know who contacted who, when, and for how long, as well as information exchanged. 
Combining the information from the different spaces, we have the possibility to deduce a causal link between 
the missions of the various actors (the actor failed in his mission because the actor B did not send the right 
information) and thus obtain a complete and precise evaluation of all the actors. 


Expert debriefing 


Decision and game result 


Collect learner 


: i Learning analysis (Performance, competence, 
information 


motivation...) 


Learning Experience 


Start Experience Observation 


Brief presentation 


End Experience 


Post game questions: 


Game instruction - Learning achievement 


- Learning outcomes 


Former observations 


Figure 3. Learning timeline 
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Global evaluation will be done at the end of the game play session. It will contain both individual 
and collective evaluation. Certification of a learner's skill or knowledge can be achieved through negotiation 
as presented in [41]. For Massive multi online games creators or game designers should take in consideration 
collaboration, culture and communication. At the end of each experience test pedagogue and professional 
expert respectively must evaluate output of game and student learning achievement. This operation is 
important for both to enhance assessment plan and to ensure homogeneity of collected data. However, this 
method is evaluated by implementing some games in our laboratory and by applying this method on some 
existing games as presented in the next part. 


6. PROOF OF CONCEPT 
6.1. Existing Serious Games 

Mecagenius[5] is an example of serious game that uses in process assessment based on the 
monitoring of states. The purpose of the game is to train engineer in mechanics, providing them with a virtual 
laboratory and various tools. The learner has to give a diagnosis and a prescription to each case of study. He 
has the possibility of running different types of tests. The system also collects secondary information for 
more formative assessment. 

Simventure[42]: is an example of business game. It allows the player’s to manage a company, 
dealing with four major types of issues: production, organization, sales and market, and finance. It proposes 
also some additional activities, such as debriefing, answering questions, writing essays, and forecasting 
events and outcomes and business planning. 

PIXELearning’s Enterprise Game[43] is a similar business game. Learner can define a product 
meeting the market demand in terms of quality and price is the most important factor to make the business 
viable. Defining their proper marketing strategy is a key as well. It’s allows to learner new aspect such as 
competition and collaboration. 


6.2. Result and Discussion 

The purpose of our game Recruitment protocol game as presented below in Figure 4 is to prepare 
the player as good as possible for his job interview or job promotion, providing him with an interview and 
various real situations. The learner should manage to face each situation by selecting right answers in the 
right moment. The player must choose among several answers the one that seems to him the most adapted, 
thus becoming aware of the rules of politeness, adaptability, motivation and so on, in order to be 
implemented during a job interview. The system also collects secondary information for more formative 
assessment. 


COM 
Sewer MEn 


Combien dinstarces de la classe A cree le 


gS code suivant); 


i 


Figure 4. Recruitment protocol 


The choice of recruitment is not arbitrary, because it combines theory and experience. Also, in the 
recruitment process we have all types of assessment, pre, in, post and global evaluation. The choice of a good 
element is not an easy job and it’s required and number of techniques and rules that we would respect to 
achieve the desired results. The designing team (pedagogue, expert and designer) must define different 
scenarios, actions, rules, data to collect and so on. When designing and developing the game we were a team 
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composed by a game designer, recruit expert and pedagogue. As mention above in our proposed assessment 
plan, actors defined the game objectives. However, the game subscribes in the recruitment and business 
domain. It has as intent Educative message broadcasting, Informative message broadcasting and training and 
it has professionals as a target. 

The evaluation of the learners in recruitment protocol requires the collection of information from 
different sources according to the role incarnated by the learner in the scenario. However, the expert and 
pedagogue have classified question into four groups, in each group of question there are several question 
with their corresponding result, choices, guidance, difficulty and so on. In the Table 10 we present a part of 
proposed question. 


Table 10. Quiz 


‘picture’: 'img/picture.jpg', 

‘group’: 'Technical skills’, 

‘question’: ' Java is a language developped by ', 
‘answers’: ['Hewlett-Packard', 'Sun Microsystems', 'Microsoft', 'Oracle'], 
‘correctAnswer': 2, 

'Son': 'disque.mp3', 

‘duration’: '312', 

‘Dificulty': '2', 

"Mark": '1', 

‘Guidance’: 'Par James Gosling chez Sun’, 
'Error': 'Please revise your lessons', 

'‘level': '1' 


The main objective of the game is to train new bachelor for a job interview and to help them to develop their 
skills required for specific jobs. The game is for computer and science student. In Table 11 we present 
student’s portfolio required in recruitment protocole. In order to achieve optimal result, we provide to collect 
a number of data as presented in the table below in Table 12. Collected Data 


Table 11. Student’s Portfolio: case of Recruitment Protocole 
student's portfolio 


Learner: is a student or employee 

Type: Junior, Senior intermediate, expert 

Tutor: Teacher, project manager 

Context: online 

Result: Score, time, number of correct answers, number of false answers, number of redundant faults, result by skills 
Note: Failed, medium, low, high, good, excellent 

Decision: pass to the next level, repeat the stage 


Table 12. Collected Data 
What specific data do we need to collect? 
Name; Age ; Sex;Time;Score; Number of gestures; Number of right answers; Number of wrong answers; 
Check if the learner need assistance or help; 
Number of redundant fault 
right action, too early, too late, suboptimal, erroneous and missing 
difficulty, 
group of question, 
question, 
answer, 
datetime_start 
datetime_end, 
score 


The pedagogue expert requires using the evaluation method capstone project considering the large 
number of advantages that it brings and it is perfectly appropriate with the nature of our project. This method 
has as purpose to integrate knowledge, concepts, and skills acquired by learner during the sequences of the 
game. Statistics and learning analytics are applied to measure, collect, analysing and reporting of data about 
learners and their contexts, for purposes of understanding and optimizing learning and the environments in 
which it occurs. 

To facilate the monitoring to the player status, there is a controle panel that allows the activation 
and/or monitoring of the following actions: enable/disable the sound, to show a number of errors, to controle 
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a number of questions in each level, to contrôle a time spent in each action, display learning debriefing at the 
end of the experience. After the players answer it, the game provides a feedback starting whether they missed 
the issue and if they spent more or less than the estimated time also if they miss a stage or some question an 

alert will be triggered. A player can demand assistance as soon as possible. The game was evaluated for a 

computer science class composed of 34 students who, after playing the game they have a detailed result about 

their experience and a debriefing about their lacks and how to develop it. Also the tutor, have a dashboard 
from them we extract the chart in the Figure 5. Especially, corporate sectors need some specific skills to 
enhance in the game or learning experience: 

a. Technical Skills: Operational the necessary skills to produce the product or service Supplies/Raw 
Materials to obtain optimal result. 

b. Managerial Skills: Management planning, organizing, supervising, directing, networking 
Marketing/Sales identifying customers, distribution channels, supply chain Financial managing financial 
resources, accounting, budgeting Legal organization form, risk management, privacy and security 
Administrative people relations, advisory board relations Higher-order thinking skills 

c. Entrepreneurial Skills: Business Concept business plan, presentation skills Environmental Scanning - 
recognize market gap, identify user needs, exploit market opportunity Advisory Board and Networking 
balance independence with seeking assistance 

d. Personal Maturity Skills: Self-Awareness ability to reflect and be introspective Accountability ability to 
take responsibility for resolving a problem Emotional Coping emotional ability to cope with a problem 
Creativity ability to produce a creative solution to a problem Risk taking, tolerance for frustration. After 
putting the game into experience we obtained this global result. 


m. 
(oe) 


E Technical Skills 


E Personal Maturity 
Skills 


E Entrepreneurial Skills 


9 
8 
7 
6 
5 
4 
3 
2 
1 
0 


E Managerial Skills 


Abdelali 
Oussama 
Ibtissam 
Mohamed 


Figure 5. Learning experience result 


The questions are grouped into four skills group technical skills, personal maturity skills, 
entrepreneurial and managerial skills. The chart presents the result of player’s experience. According to 
obtained result, the former or the company has a visibility to their employee’s competencies. Also, the 
learner, trainee or employee has a clear idea about their lack and skills to develop to have better career 
evolution. The results as presented in the Figure 5 shows that the majority of players have a good level in 
technical skills and personal maturity skills but they still have to improve others skills. 

At the moment, this assessment strategy proved successful. We hope to test it in a large wide to 
validate it and to ensure better integration of assessment in the future for both the professional and 
nonprofessional. The main objective of an evaluation plan is to evaluate, in general terms, the viability of all 
pedagogical aspects of a serious game, including the description and analysis of development prospects. 
Regardless of the size of the project, the evaluation plan is an essential step for anyone seriously wanting to 
create their own serious game. As this step is too often overlooked, our research team has decided to simplify 
things by proposing a simple and effective structure to build an evaluation plan tailored to the needs of the 
designer. 
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According to this study we can define easily needed steps and parameters to integrate assessment in 
the game design process, in order to create a beneficial strategy of assessment that will help the learners to 
improve their level and accomplish their learning achievement, in addition it will make the serious game one 
of the great tools for learning assessment. 

This project is a special device to accompany the creators of the game, from the stage of preparation 
of the story and preparation of the project until the end of preparation of an evaluation plan. This project was 
designed to help decision-makers or educators ensure the sustainability of their serious games. Highlight of 
this approach are: 

a. The simplicity of expressing needs. 

b. The framework can be exploitable by both for professional and non-professional people 
c. The possibility of integrating the elements of quantitative and qualitative evaluation 

d. Gives a clear and detailed idea of the project 

In this step of our project we provide to collect the necessary information about the evaluation plan. 
As a first step, we have integrated the quantitative evaluation considering the different advantages that they 
present. In future work, we will integrate assessment intangible measures as the motivation, behavioral 
changes, and learner performance. 


7. CONCLUSION AND FUTURE WORKS 

The assessment in serious games is one of the important mechanisms that push learners to achieve 
their learning objectives; therefore, it makes the serious games one of the effective learning tools. During this 
paper, several assessments techniques and methods applied on serious games have been discussed. Among 
the techniques used by our research team there are the matrix of assessment and data collection, they have 
been applied on some kinds in the game developed by our team of research recruiting protocol, according to 
this experience a new scheme for a proper integration of assessment in serious games, supporting different 
methods and tools in different steps of development process has been developed. The proposed schema will 
be improved thereafter, in order, to enhance the platform that will help the game designer to establish some 
assessment strategies in a very simple way, respecting the acquired and the learning outcomes envisaged by 
the experts. 
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